Applying the Rasch Model by Trevor Bond Christine M. Fox & Christine M. Fox

Author:Trevor Bond,Christine M. Fox & Christine M. Fox [Trevor G. Bond]
Language: eng
Format: epub
Publisher: Taylor & Francis Ltd

Extended Understanding—Chapter 8

Invariance of Rated Creativity Scores

So, having completed the MFRM analysis of the judgments of junior scientists’ creativity ratings, is everything ‘hunky-dory’, as they say in the classics? Well, no. If we return to take the principle of measurement invariance (from Chapter 5) seriously, we would require that the creativity estimates of junior scientists should remain invariant across judges (especially after we have already taken into account the differences in judge severity). In Figure 8.4, taken directly from the FACETS output, we have plotted the creativity of each junior scientist as judged by senior scientists A and C (less severe) on the horizontal axis against that candidate’s estimate as judged by senior scientist B (more severe), alone. The modeled identity (or invariance) line is dotted in that FACETS figure.

Estimated measures of junior scientist creativity are not invariant across senior scientist ratings. Clearly, while Judge B is marginally more severe than the others, it is more important to note that Judge B’s ratings vary too much from those of Judges A and C to be considered part of the same measurement system. We saw a hint of that in Table 8.2: While Guilford (1954) concluded that judges A and B each produced one of the most unexplained ratings in his analyses (indicated by the superscript g in Table 8.2), the MFRM analysis attributed all five of the most unexpected ratings to Judge B (indicated by the superscript f). As a judge, B’s performance is too idiosyncratic. Its seems that the Navy examinations board should have seriously considered retiring Senior Scientist B from the scientific creativity judging panel. The control file for this many-facets Rasch analysis is given at the end of Chapter 8. More detailed consideration of this analysis is given in chapter 10 of Linacre’s (1992) expository text on the many-facets Rasch model.

Figure 8.4  Judge B’s candidate measures plotted against those of Judges A and C together. An approximate expectation line is drawn in. Points are identified by junior scientist code (a–f ).

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.